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1.  Introduction 

Let  (X^,  t  =  0,  +  1,  +  2,  . . . )  be  a  discrete  parameter  time  series 
generated  by  a  model  of  the  form 


Xt  =  mt  +  Yt 


■where 


\  -  E<V  = 


(A  cos  cj  t  +  B  sin  a)  t)  , 

'  VI  "M  "M  '  ' 


with  0  <  a)  <  Jt  (which  involves  no  loss  of  generality),  and 


(1.1) 

(1-2) 
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00 
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u=0 
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(1.3) 


the  £_£  being  distributed  identically  and  independently  each  with  mean 
zero  and  finite  variance  E(e^)  =  v,  and  the  g  (£)  being  specified  func¬ 
tions  of  an  unknown  vector-valued  parameter  Q  =  (0^, 8^, • • . ,6^)  such  that 
Xu-0  Su(0)  <  °°*  bo  av°l^  indeterminacy  we  take  g  (d)  =  1.  (X^_)  thus 

has  a  systematic  component  consisting  of  the  sum  of  q  simple  harmonic 


components  with  different  frequencies  and  a  residual  or  'noise'  com¬ 

ponent  which  is  a  completely  stationary  series  having  spectral  density 


f(oue)  =  ’  (v/2rt)  |  2  S..(e)  elWU|2  ,  (1.4) 

u=0  ~ 
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and  is  usually  called  a  linear  process  (see,  for  example,  Hannan,  i960, 
P-  33). 

Suppose  that  the  values  of  the  other  parameters  in  the  model, 
namely  A^,  B  ,  10^  (l  <  r  <  q)  and  v,  are  also  unknown.  We  then 
have  a  fairly  general  type  of  'hidden  periodicities'  model,  the  term 
'hidden  periodicity'  denoting  a  harmonic  component  whose  frequency  as 
well  as  its  amplitude  and  phase  is  unknown.  The  restricted  model 
obtained  by  taking  the  residual  component  to  consist  of  'white  noise', 
that  is,  SUC£)  =  0  for  u  >  1,  so  that  (1.3)  just  becomes  Y^  = 
and' the  parameter  .  disappears,  has  been  quite  widely  used  in  connec¬ 
tion  with  physical  and  economic  phenomena,  though  in  recent  years  it 
has  become  less  popular  because  of  the  realisation  that  analyses  based 
on  this  can  be  very  misleading  if  the  'white  noise'  assumption  is  not  a 
good  approximation. 

The  problem  of  .estimating  the  parameters  in  the  model  (l.l)  from 
data  consisting  of  n  observations  X^  = 1  (X^,Xg,  . . .  ,X^) ,  and  of 
determining  the  approximate  distribution  of  the  estimators  for  large  n 
was  dealt  with  by  Whittle  (1952).  He  used  a  method  of  estimation  which 
was  approximately  equivalent  to  an  application  of  the  principle  of 
least  squares,  becoming  approximately  the  method  of  maximum- likelihood 
when  et  has  a  normal  distribution  so  that  (X^)  becomes  a  normal  or 
Gaussian  process.'  By  means  of  heuristic  arguments  he  obtained  the 
following  results. 

/S  /S  A 

(l)  The  estimator  (A_^,B^,w^)  is  asymptotically  (n  -*•  °o)  normal 
with  mean  0  and  covariance  matrix 


•3- 
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(1-5) 


■where 


g(a),e)  =  j  £ 


u=0 


/Qv  iwui2 

gu(£)  e  | 


(1.6) 


See  Whittle,,  1952,  p.  53,  equation  (4.14)  and.  1954,  p.  224,  equation 
(ll).  Note  that  the  change  in  sign  of  the  first  two  elements  in  the 
last  row  and.  last  column  in  (1.5)  is  due  to  Whittle  having  interchanged 
A_^  and  B^  in  (1.2),  and  that  his  computation  of  the  bottom  diagonal 
element  is  incorrect,  the  numerical  factor  being  l/3,  not  l/6. 

(2)  (A]_,B-j_,u^) ,  (Ag ,  Bg ,  ) ,  ...  ,  (A^,B  ,<»>  )  and  _£,  the 
estimator  of  0,  are  all  mutually  asymptotically  uncorrelated. 


(3)  0  is  asymptotically  normal  with  mean  0  and  covariance 


matrix  (riW) 


where  the  elements  of  W  are  given  by 


1 


d  log  g(i*),0)  d  log  g (co,  0) 


WiJ  ^  J_n  5  ®i 


5  0. 
3 


dto 


^  (constant  term  in  expansion  of  {log|  S,n(S)zu\^} 

i  u=0 


ui2-, 


■S.  OO  „  / 

X  gg-  {logj  ]T  gu(£)zU|  )  on  the  unit  circle) 

°  j  u=0 


See  Whittle,  1952,  p.49,  equation  (3*7),  1954,  p.2l4,  equation  (7) 
The  estimators  are  obtained  by  minimising  the  expression 


(1-7) 
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n 


E  V-)Cs  “2n  .  £  h(cor,S)  ,J]X  (A  cos  id  t  +  Br  sin  id  t) 

s  <  n- 1  °  "r=l  t=l 


(1.8) 


+  I  ^  h(“r^)  <Ar  +  B")  >' 


where  C  =  n  ^  X  X,  i  i  (0  <  s  <  n-l)  are  the  sample 

S  X  X  *  |  S  I  '  1 

covariances  (with  divisor  n) ,  h((d,j3)  =  {g(u,e)}_1,  and 


aB(fl)  =  ^  f  eltJS  h(w,e)  did  .  (1.9) 

>0  -It 

Compare  Whittle,  1952,  P*50^  equation  (4-5)  or  1954,  p.22J,  equation 

(j).  These,  apart  from  a  factor  n,  are  the  same  as  (1.8)  when  the 

expression  for  Q!  (0)  given  by  (1.9)  is  substituted  in  the  latter, 
s 

Our  object  will  be  to  present  proofs  of  these  results  stated 
precisely  as  limit  theorems,  under  conditions  which  should  be  satisfied 
in  nearly  all  applications.  These  follow  the  usual  pattern,  whereby 
consistency  of  the  estimators  is  first  established,  and  then  the  mean 
value  theorem  applied  to  obtain  asymptotic  normality  with  the  aid  of  a 
central  limit  theorem.  However,  great  care  needs  to  be  taken  with  the 
details,  especially  in  the  consistency  part  of  the  proof.  It  might  be 
thought  that  a  simple  modification  of  the  argument  used  in  a  previous 
paper  (Walker,  1964)  to  give  a  rigorous  proof  of  the  result  (5)  above 
when  no  harmonic  components  are  present  would  suffice,  but  the  presence 
of  the  unknown  frequencies'  makes  the  situation  a  great  deal  more  compli¬ 
cated.  In  the  'white  noise'  case,  X^  =  m^_+e^,  the  complications, 
although  still  quite  troublesome,  are  substantially  reduced.  We  there¬ 
fore  deal  first  with  this  simpler  situation  in  the  present  paper.  The 
general  model  with  residuals  generated  by  a  linear  process  will  be 
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considered  in  a  subsequent  paper.  Appreciable  simplification  is  also 
achieved  when  there  is  only  one  harmonic  component  present.  For  this 
reason,  we  shall  take  q=l  in  (1.2)  in  the  proof  that  follows  in 
§§2-4,  and  then  indicate  the  modifications  required  when  q>l  in  §5, 
the  final  section  of  the  paper. 
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2 .  Consistency  of  the  estimator  of  angular  frequency. 

For  q=l  it  will  be  convenient  to  make  a  slight  change  of  notation, 
and  write 


m^_  =  A  cos  cot  +  B  sin  cot 


(2.1) 


In  the  'white  noise'  case,  h(co,£?)  =  1,  and  so  (1.8)  reduces  to 


C  -  2n'1  Y,  xt  (A  cos  ^  +B  sin  lot)  +  ~  (A^  +B^)  .  (2.2) 


n 


1  /  »2  ,  t,2. 


t=l 


Minimisation  of  (2.2)  with  respect  to  A ,  B,  and  co  then  gives 
estimators 


A  =  —  Y  X ,  cos  cot  , 
n  n  t 


n 


t=l 


B  =  -  V  X  sin  cot  ,  (2.3) 

n  n  t  ’  K  r  ' 


and  co  such  that 
n 


I  (co  )  =  max  I  (to)  , 

n  n  rs  ^  ^  n'  '  ’ 

0  <co<  jt 


(2.4) 


where 


I  (co) 
nx  ' 


o  n 

=  !  ii  xt e 

t=i 


icot  1 2 


(2.5) 


the  usual  definition  of  the  periodogram  intensity  function  (see,  for 
example,  Hannan,  i960,  p.52).  The  suffices  n  have  been  added  to 
emphasise  that  the  estimators  depend  on  n;  strictly  we  should  write 
Ap(X^n^)  etc.,  but  the  omission  of  the  argument  will  cause  no 


ambiguity. 
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Now  (2.2)  is  approximately  equal  to  n  ^  S(A,B,w),  where 

n  p 

S(A,B,to)  =  (X  -  A  cos  wt  -  B  sin  wt)  (2.6) 

t=l  t 

is  the  residual  sum  of  squares,  provided  that  00  is  not  near  0  or  it. 
For 

n 

Z  (A  cos  wt+B  sin  wt)^  -  ^  (A?  +B^) 
t=l  ^ 

,  n 

=  ±  £  ((A  -B  )  cos  2wt+2AB  sin  2wt] 

t=l 

and 

£  (cos  2wt  +  i  sin  2«t)  =  ei(J(n+1)  j_s  o(l) 

if  10  lies  in  a  closed  interval  contained  in  [0,it].  Thus  our  estimation 

k 

procedure  should  be  a  reasonable  one  if  the  true  value  of  00,  say, 

is  not  0  or  it. 

From  now  on  we  therefore  make  the  assumption 

0)Q  f  0  or  it  ,  .  (2.7) 

which  is  quite  a  mild  restriction.  (2.7)  would  not  be  needed  if  we 

/S  /*N  /% 

were  to  determine  Ar,  B  ,  oo^  by  minimising  S(A,B,oo)  exactly,  but 
this  leads  to  much  less  pleasant  estimation  equations  which  are  extremely 
inconvenient  for  theoretical  investigation  of  properties  of  the 
estimators . 

When  is  normal,  the  log- likelihood  function  is  of  course 

1  n  „  S(A,B  ,00) 

=  -  2  n  log  2itv  • 


Ln(A,B,w,v) 


(2.8) 


-8“ 


AAA 


*aus  A^,  B^j  are  approximately  maximum- likelihood  estimators, 

being  obtained  by  maximising  a  modified  log- likelihood  function  given 
by  replacing  n  ^  S(A,B,to)  in  (2.8)  by  (2.2). 

Since  •  to  is  determined  by  maximising  In(to)  it  is  natural  to 
show  first  that  this  is  consistent.  We  shall  in  fact  obtain  a  much 
stronger  result,  without  which  it  is  not  clear  how  to  establish  con- 

/\  /N 

sistency  of  A  and  B  .  . 

n  n 

Theorem  1.  Let  m^  =  A^  cos  to^t  +  B^  sin  to  t  where  to^  f  0  or 

A  A 

Then  if  to  is  such  that  I  (to  )  =  max  I  (to), 
n  n  n  ^  ^  ^  n 

0  <  to  <  it 


to  -  to  =  o  (n  j  , 
n  0  p'  ' 


n  ->  oo 


(2.9) 


so  that  in  particular  to^  is  consistent. 
Proof 


Prom  (2.5)  we  have 


5  “ 


11 

t=l 


eltJt  (aq  cos  toQt+B0  sin  toQt+et)|2  .  (2.10) 


Write 


itoQt  -ito  t 

Aq  cos  toQt  +  Bq  sin  toQt  =  DQ  e  +  D*  e 
where  DQ  =  ~  (Aq-  iBQ) ,  D*  =  ^  (A0  +  iBQ).  Then  (2.10)  gives 


2  “  In(“) 


|Dq  Mn(to+to0)  +D*  Mn(to-toQ)  +  et  e 


itot  1 2 


,  (2,11) 


where 
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n 


M  (u) 
n 


z  - 

t=l 


iut 


=  e 


^  i(n+l)u  ^sin  ^  nu' 
.  .  1 

sm  g  u. 


0  <  u  <  2it 


,(2.12) 


=  n 


u  =  0  or  2jt 


that  is, 


\  n  In(w)  =  |  ^  ete1“t|2  +  2^{(^  eteiart)(D0  Mjco+a^)  +  D*  Mn(co-coQ)) ) 


+  |DQ  Mn(co+u0)  +  DSMn(n)l  * 


When  co  =  co^,  (2.13)  is  dominated  by  the  term 


(2.13) 


D* 


»Mn(0)r  -  ^2(A20*BS0) 


iu>0t 


In  fact,  since  the  real  and  imaginary  parts  of  e^.  e  each 

have  variance  ^  nv  +  0(l),  so  that 


n  ico  t  . 

V  e  e  u  =  0  (n '  )  ,  and  M  (2<oJ  =  0  (l) 

t  P  nv  0  p 


because  of  the  condition  co^  f  0  or  it,  we  see  that 


|nIn(“0)  =  irn2(An+Bn)  +  «„(2/2)  , 


0  O' 


or 


W  =  !"(Ao+B?>  +  °p("l/2)  • 


(2.14) 


We  now  obtain  an  estimate  of 


max  I  (co),  where  5 
_  n'  ’ 

i  w-w0 1  >  n  ^ 


can  be  arbitrarily  small.  For  this  we  require  the  inequality 
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El  max  |  £  e,  <  v{n  +  2  P  x1/2  dx)  .  (2.15) 

0<w<jr  t=l  1  J 1 


This  follows  since 


n 


II 


£t  6 


iorfc  1 2 


E  -1MS 

s  <  n 


n-  s 


J-OUko  r~i 

-ie  t-i  s*  et+i 


n-  |  s  | 

<  I  I  [  e+  e  ._j 

s  <n-l  t-1  •  t  t+l' 


whose  expectation  does  not  exceed 


n  ^  n-l  n-s 

*  (Z  et}  +  2  2  Et(  2  et 

t=l  s=l  t=l 


.2^1/2 

W  5 


n-l 

v{n  +2  I 
s=l 


(n-s)1/2} 


(Compare  Walker,  1965,  p.112,  equation  (29)).  From  (2.15)  we  have 


max  |  V  €,  eiWt|2  =  0  (n^/2)  .  (2.1 6) 

0  <  w  <  rt  t=l  *  P 


Using  (2.l6)  and 


max  |M  (u+oj  )|  =  0(l)  ,  max  I M  ( co  -  co  )  |  =  n  , 

0 <  (J <  rt  u  0  <co<  it 


we  see  from  (2.13)  that 


max  |  r  nl  (u)  -  |d*M  (uj-u>  )|2|  =  0  (n5/2)  +  0  (n7^)  +  0(n)  , 

1  2  '  '  0  nv  O'1  pv  '  v  ' 

0  <to  <  it  ^ 


giving 


max  |ln(co)  -|  n_1(A2+B2)  ]Mn(u-a)0)|2|  =  0  (n5/4)  .  (2.17) 

0  <  w  <jt  '  e 
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If  we  were  to  add  the  assumption  that  E(|e,|r)  <  «>  for  some 

"t> 

r  >  we  could  use  a  much  more  powerful,  but  by  no  means  elementary, 

pesult  of  Whittle  (1959;  p.l80,  equation  (¥0),  according  to  which  the 

factor  rr'  ■  in  (2.16)  can  be  reduced  to  n  log  n,  so  that  ( 2.17 ) 

l/2 

becomes  Op{(n  log  n)  ):>  but  this  is  not  necessary. 

2  2  1  2  1 

Wow  the  function  |M  (u)]  =  (sin  —  nu)/(sin  u),  0  <  u  <  2rt 


2  2 
2 


decreases  steadily  from  its  absolute  maximum  of  n  at  ,  u  =  0  to  a" 
minimum  of  zero  at  u  =  2it/n.  For  the  derivative  of  log  |m  (u)|2  is 


n 


1  1 

n  cot  —  nu  -  cot  —  u  =  (2/u)  {^(nu)  -  ij/(u)}  , 


where  \[r(x)  =  x  cot  x,  and 
1  2 

\jr'(x)  =  cosec  x(-sin  2x-  2x)  <  0  when  x  >  0 

Hence  for  any  prescribed  6,  such  that  (sin  ^  6  / ^  S)2  >  l/jt2. 


max  |Mn(m-u0)|' 
|  oj-oj01  >n-15 


.  2  1  _ 
sin  —  6 

2  1  -1_ 
sm  g  n  6 


(2.18) 


when  n  is  sufficiently  large,  since  further  local  maxima  of  this  function 

2  it 

must  be  less-  than  cosec  —  .  It  follows  from  (2.17)  that 

n 


max 

|  CO— a)  I  >n  *4) 


<  \  n-\AZy0) 


.  2  1. 

Sin  r-  O  \  /, 

.  mr)  ♦  0p(n5A) 

vsm  rr  n  by 


,  .  1  ~\2 
i  p  p  /  P  8 

■  bK<>(  — 


■  1  "I  RJ 

gin  —  n  0/ 


+  0  (n5/4) 

P 


(2. 


if  n  >  n^S),  say.  Hence 


19) 
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p  11m 


n  -*■  oo 


max  I  (to)3 
*  1  n 

1  co-Co  1  >n-18 


<  \  (Ao+Bo)  =  p  lim  ^n_1 

n  ->'<» 


from  (2.1^).  This  implies 


lim  P{  max  I  (to)  <  I  (to_) }  =  1  . 

,  nv  n'  0 

|co-Wq|  >n  BS  , 


and  therefore 


lim  P{n|ton-to0|  <  83  =  1  .  (2.20) 

n  -»  oo 


As  8  can  be  arbitrarily  small,  (2 .20)  is  equivalent  to 


to  -  to 
n  0 


=  o  (n_1) . 
.  P 
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3 •  Consistency  of  estimators  of  coefficients  of  sine  and  cosine  terms. 

We  had  A  =  -  X.  cos  u  t,  B  =  -  .  X.  sin  co  t .  (2.3) 

n  n  t-1  t  n  ’  n  n  t=l  t  n  v 

Theorem  2 .  Under  the  conditions  of  Theorem  1, 

/N  /S 

p  lim  A  =  and  p  lim  B  =  Brt  (3-1) 

nO  nO  / 

n  ->  00  n  -*•  co 


Proof 

Prom  the  definition  (2.3), 


/\  /S 

A  +  i  B 
n  n 


0  n  im  t  -iw_t  iw  t 

n  ^  <D0  6  +D0e  +et)  6 


and  so 


(A  -A_)  +i  (B  -B  ) 
v  n  0'  v  n  0 


o  a  n  ito  t 

-  n  WW  +DS<M„(V“0^  +  X  €t  e  1 


Thus 


•|  (A  -A  )  +i(B  -B.) 
1 '  n  0  '  n  0 


2|D  I  ^  ~  o  n  iw  t 

<  - —  { |M  (w  +00  )  j  +  |m  (u>  -w_)-n| )  +  -  I  f  e.  e  I 

—  n  1  nv  n  O'  1  1  nv  n  0  1  n  t  1 


(3-2) 


Since  |(jd^-conj  <6  is  equivalent  to  2w^-6  <  w^+wn  <  2mA+S, 


n  0 

/N 

consistency  of  clearly  gives 


n  0  0 


p  lim  n  M  (u)  +  w_)  -  0 

i  nN  n  0  1 

n  ->  00 


(3.3) 


Also 
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M  (to  -  to.)  -  n  =  M  (to  -  toj  -  M  (o) 
rr  n  0  nv  n  0  nv  ' 


=  (to  -  to  )  M'  (to*) 
n  O  n  n 


(3-4) 


where  to*  g.  (to  ,to  ),  by  the  mean  value  theorem.  But  for  all  to. 

n  0  n  ’ 


n 


iM^to)  | 


,  itot .  2 

te  <  n 


t=l 


Hence 


that  is, 


—  1  ^  ^ 

In  (M  (to  -to  )-n)  |  <  n|to  -  to  I 
1  v  nv  n  0'  '  1  1  n  01 


0  (4)  by  Theorem  1  , 


p  11m  |n  1(Mn(tOn-w0)  -n)  |  =  0  . 


n  -*  oo 


ito  t  , 

Finally,  from  (2.16),  1^^  e  n  |  =  and  so 


(3-5) 


n 


P  lim  n  I Z 


n  '  /->  *t 
n  -»  oo  t=l 


ito  t 
n  i 

e  =o 


(3.6) 


(3.3),  (3- 5);  (3-6)  thus  give 


p  lim  |  (A  -A  )  +  i  (B  -B  )  | 
1  '  n  0  n  0 


n  ->  co 


0 
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4 .  Asymptotic  distribution  of  the  estimators. 

Define 

n  o  n 

U  (A,B,w)  .  =  Y  XT  -  2  Y  X,  (A  cos  rnt  +B  sin  wt)  +  —  n(A  +B  )  ,  (4.1) 

n  t=AL  t=l  X  d 


■which  is  equal  to  the  approximation  to  the  residual  sum  of  squares  that  is 

A  A  A 

minimised  to  obtain  the  estimators  a)  ,  A  ,  B  .  Writing 

n  n  n 


au 


a2u 


oA 


r  =  ^Un^A-’  §ASm  =  ^Un^Am-’  etc?  ve  haye’  the  mean  value  theorem. 


(Un^A  ^Un^A*A*^A0“An^  +  ^Un^A*B*^B0"Bn‘>  +  (Un^A*w*^“cf  “n^ 
0  n  n  n  n  n  n 


(4.2) 


(VB0  ^Un^A*B*^A0“An^  +^Un^B*B*^B0"Bn^  +  ^Un^B*c^^“o"Wn^  >' 


n  n 


n  n 


n  n 


and 


<V«L  “  <Dn  W<W  +  <Dn  W<W  +  <°n  W  VV 

0  n  n  -  “  -  - 


n  n 


n  n 


(4.4) 


•where  we  use  the  generic  notation  (An'Bn;t°rP  for  a  P°-*-n^  on  He  line 

AAA 

joining  (AQ,B0,u0)  and  (An^Bn^wn) t  so  that 

(A*,BS^)  =  ^(A0,B0,U())  ♦  (l-A)(An,Snln)  ,  ,  (U5) 

where  0  <'  K  <  1.  The  fact  that  the  points  (A*,B*, w*)  in  (4.2), 
(4.3)  and  (4.4)  are  different  will  cause  no  ambiguity. 

Wow 
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(Va, 


=  n  A. 


0 


n 

& 


X^_  cos 


V 


n 


=  nA0  “ 


Y  (A  cos£ 
t=l  0 


w_t+B,_  sin  03_t  cos  w  t  +  6 ,  cos  cj_t)  (4.6) 
00  0  Ot  0 


=  -  2 


n 

t§x 


cos  +  0(l) 


Similarly 


n 

(UnL  =  -  2  Y.  et  sin  V  +  °(1)  '  (4-7) 

n  *0  t=l  *  0 


and 


<VU, 


0 


The  sums,  in  (4.6) 


n 

2  Y  e.  t(A  sin  wAt  -Bn  cos  tx>  t)  +  0(n)  .  (4.8) 

*t  ^  v  u  u 

-  (4.8)  are  all  of  the  form  k^.  where 


lim  max  lk+l/(])b  k^)4^2  =  0  .  (4.9) 

n  ->  oo  l<t<n  t=l 


For  example,  with  (4.8) 


£  k2  =  4  £  t2{|  A2(l  -  cos  2wQt)  +|  B2(l  +  cos  2uQt)  -AQB0  sin  2uQt) 
"fc  — 1  "fc=l 


=  2(A2+B2)  £  t2  +  0(n2)  =  |n3(AQ  +  B2)  +  n2  . 


t“l 


It  follows  that  the  central  limit  theorem  will  apply  to  these.  For  (4.9) 
implies  the  Lindeberg  condition  (see,  for  example,  Rao,  1965,  p . 108 ) , 
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since,  writing 


n 


k*  et  > 


a  =  (v  y1  k.) 
n  t=i  *’ 


2,1/2 


and  G.  for  the  distribution  function  of  t) ,  we  have 
X  •  X 


n  p 


r|  |  >  8a 


n2  dGt(ri)/a^ 


n 


_1_ 

a2  t=l 
n 


£  k2  r 

&L  t  J  I 


ej  >8an/|kt 


dF(e) 


where  '  F  is  the  distribution  function  of  e^,  and  so  does  not  exceed 

/ 1  -,i  ^  5*  /  I,  i  e  dF(e),  which  tends  to  zero  when  n  -»•  <».  Thus 
J  |  e  |  >  Sa  /max |  k,  |  '  '  ’ 


rV2, 


r1/2/ 


r3/2 


-  n  '(UjB  ,  n  (UjM  converge  in  law  respectively  to 


0 


n  B 


0 


n  w 


0 


N(o,2v),  N(0,2v),  and  N(o,|  (A2+B2)v)  when  n  -  ». 

For  the  limiting  joint  distribution  we  consider  the  random  variable 

Vh'W  =  h  "'1/S(Va„^  n‘l/2(Un)B  +  k*  n'5/2(Uh-  - 


"0  “  “  ~o  5  "n'wo 


the  are  arbitrary  real  numbers.  Now 


JG,  --5  /p 

vn  =  2  £  et{x3  n  t(A0  sin  V  -Bo cos  wot) 


-  n"1/2  (k^  cos  wQt +kg  sin  wQt)}  +  0(n_1^2)  ,  (4, 


and  the  sum  in  (4.10)  is  of  the  form  k  ,  e,  where 

X— X  n,x  X 


lim  max 
n-*oo  l<  t  <  n 


kn,t^ 


2  ^l/2  _ 


=  0 


(4.11) 


— 1/2 

the  numerator  in  (4.1l)  being  0(n  '  )  and  the  denominator  0(l) . 

Hence  the  central  limit  theorem  will  apply  to  this  sum  also  by  using 
the  generalised  Lindeberg  condition  (see,  for  example,  Loeve,  i960, 
p.295),  which  is  implied  by  (4.1l)  in  exactly  the  same  way  as  the 


10) 
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ordinary  Lindeberg  condition  is  implied  by  (4.9) «  Now  a  straight¬ 
forward  calculation  shows  that 


n 


lim  Z  kn,t  ’  +  |<Ao  +Eo»f  +  2W3  '  2Ao¥-3  ■  (4-12) 

n  -»  oo  t=l  * 


Thus  Vn  converges  in  law  to  a  normal  distribution  with  mean  zero  and 

variance  (4.12).  Consequently ,  by  using  the  equivalence  of  convergence 

in  law  and  pointwise  convergence  of  characteristic  functions,  (see,  for 

example,  Rao,  1965,  p.103),  we  see  that  the  joint  distribution  of 

n"-*-/^(u  )  ,  n  (U  )„  ,  n  ^/2(U  )  converges  in  law  to 

'  n  k  n  Bq  '  n  0) 


W( (0,0,0),  2vW) 


(4.13) 


where 


¥  = 


1 

0 

!bo 


0 


lAo 


(4.14) 


Next,  we  look  at  the  behaviour  of  the  second  partial  derivatives 
occurring  on  the  right-hand  sides  of  (4.2)  -  (4.4).  Three  of  these 
require  no  analysis,  as 


(Un^AA  ^Un^BB  n  ' 


'Un^AB  *  °  * 


(4.13) 


Now  (U  ),M  =  2  X ,  t  sin  art,  and  so 

x  n  '  AM  1  +•  ? 


n  Aw 


t=i  t 


n 

^n^or*  =  2  I  (A0  COS  U0t+B0  Sln  “0t+€t)t  Sin  “n1 
n  n  t=l 


n 


n 


B q  £  t  cos(to^-wo)t +2  Y,  6-t't  sin  w*t+0(n) 
U  t=l  n  t=l 


(4.16) 
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From  Theorem  1,  to  -  to.  = 

n  0 

to*  -  to  =  o  (n  ■*")  .  Hence 
n  0  P 


o^(n  ),  and  so,  from  (4. 5),  we  also  have 


n  i(to*-to  )t 
Y  t  e  .  =  M‘  (to*  -  to  ) 

t=l  n  n  0 


=  M'(0)  +  (to* -to.)  Mn{X(to*-toJ}  , 
n  v  n  0  n  '  n  0  ’ 


0  <  \  <  1  , 


=  |n(n  +  l)  +  °p(n2)  > 


since  |M"(to)  |  <  t2  ~  ^  rP  for  all  to,  and  so 

n  — “  X— X 


p  lim  n~2  Y  t  cos(to*-  toQ)t  =  ±  . 

n  -»■  oo  t=l 


(4.17) 


Also,  employing  an  argument  similar  to  that  following  equation  (2.15) 
in  §2,  we  have 


IS. 


itot  1 2 


t=l 


I  -Il0s 

s  <n 


n-  s 


_x  e  -  ^  et  et+|s| 


,  n-  |  s  J 

1  E 

|  s  I  <n-l  t=l 


£  .  .E  X,  et  et+|s|  tft+l6!)!  . 


so  that 


n 


E{  max  |  Y  €t'fceiWi:|2) 


0  <  to  <  it  t=l 


<  E(J  t  z\)  *Z  l  [E((‘ “ff  «t  €  t(t+S))2)]l/2 

t=l  S=1  t=l 


=  |  n(n+l)  +  2  Y  {  £  ^(t+s)2)1/2 

2  s=l  t  =1 

<  —  n(n+l)  +  2(n-l)  n^2  <  ^ 


(4.18) 
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((4.18)  is  a  rather  crude  inequality.,  but  it  suffices  here,)  It  follows 
from  (4.18)  that 

|  Y  e.t  sinoi  tj  <  {  max  |  £  e,  te1Wt  j2}1/2  =  0  (xJ^)  ,  (4.19) 

t-1  n  0  <oj<  it  t=l  t  p 

Hence,  from  (4.l6),  (4.17),  and  (4.19)., 

p  lim  n"2(Un)A^  =  .  (4.20) 

n  -*■  co  n  n 

Similarly,  since 

n 

(Un)BV  =  -2  £  (A0  COS  V  +B0  sin  V+et}t  cos  ^ 

n  n  "fc  — ‘1 

n  n 

=  "An  Z  t  cos(u£-w  )t  -2  £  e.t  cos  w*t  +  0(n)  , 

U  t  =1  n  U  t=l  t  n 

we  obtain 


p  lim 
n  -»  00 


-2 

n 


^n^B*w* 
n  n 


±A 

2  0 


Finally, 


(4.21) 


(Un^w*oo* 
n  n 


=  2 


n 

I 

t=l 


X,  t  (A*  cos  co*t+B*  sin  w*t) 
t  n  n  n  n 


cos  co^t  +  Bq 


sin  co  t)(A*  cos  co*t  +  B*  sin  co*t) 
0  n  n  n  n 


n 


+  2 


A*  )  e,  t 

ntti  * 


cos 


co*t 

n 


+  2 


B* 

n 


n 

k 


%  t 


sin  co*t 
n 


(4.22) 


We  can  show  that 
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E{  max  e  t2elWt]2}  =  o(n11//2) 

0  <  to  <  it  t=l 

in  exactly  the  same  way  as  we  obtained  (4.18).,  and  hence,  (4.5) 
consistency  of  A*  and  B*,  that  the  last  two  terms  of  (4.22) 
each  0^(n'12",//^) .  Also  the  first  term  of  (4.22) 

n  P  '  P 

=  y  t  {A  A*  cos(to*-to  )t  +B.B*  sin(to*-to  )t}  +  0  (n  ) 
^  On  '  n  0  On  \  n  O'  ' 

■  -  y  2(a^b")  +  op(nh 

since 


n  i(co*-to  )t 

£  t2  e  n  0 
t=l 


M"(to*  -  toj 
nv  n  0 


Hence 


Thus  if 


M"(0)  +  (to*-toA )  M"{\((o*-wA ))  ,  0  <  A 

n'  n  0  n  n  0  7 


p  “'Xw  -  y  <Ao  +Bo)  • 

n  ->  co  n  n 


¥* 

n 


n  ^Un‘>A*A* 
..  n  n 

n-1(U 

n  A*B* 

-2  n  n 

n  (Un^A*w* 
n  n 


n  (uJa*b* 

.  n  n 

n  (Un^B*B* 
_  n  n 

n  (Un^B*to* 
n  n 


n~2(u„W 

_P  n  n 

n  ^Un^B*to* 
z  n  n 

n  (Un^to*to* 
n  n 


we  have,  from  (4.15),  (4.20),  (4.21)  and  (4.24), 


p  lim  W*  = 
n 

n  -»  oo 


giving 

are 


(4.23) 


<  l  . 

(4.24) 


(4.25) 


w  . 


(4.26) 
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We  can  now  easily  establish  the  following  result. 


Theorem  3 •  Under  the  conditions  of  Theorem  1, 

{n"^(A  -A  ),  n"^2(B  -B  ),  n^/2(w  -on)}  converges  in  law  to 
n  u  n  u  n  u 


N(0,  2vW-1)  , 


(4.27) 


when  n  -*  oo,  where  0  denotes  the  row  vector  (0,0,0) 
Proof .  Writing  (4.2)  as 


a'l/2(Un>An  =  “'XW*  “1/2(A0-Jn)  +  n’X)  ^(B^) 

0  n  n  n  n 

+  n"2(U  )  n3/2(w  -w  )  , 

n  A*w*  On 

n  n 


and.  (4.3),  (4.4)  similarly,'  we  see  that 


{n"l/2(U  )  a  n-^(U)B  ,  n"^c(U  )  ) 

n  Aq  n  BQ  n  o)Q 


rVs, 


,-3/2, 


=  -  {n1/2(A  -A.),  nx/^(B  -Bn) ,  -to  ) }  W* 

'  n  0  '  n  O' '  '  n  0  n 


1/2, 


,3/2, 


Thus 


(nl/2(in-A0),  n1/2(Sn-B0),  -X  VV  J 

■  ■■  f“‘V2(Vv  I"!*'1  •  (4-28) 

Hence,  as  {n  "^(U^)^  ,  n  *^2(Un)g  ,  n  ^/2(Un)u  )  converges  in  law  to 

N(0,  2vW),  equation  (4. 13),  (4.27)  follows  from  an  obvious  generalisa¬ 
tion  of  an  elementary  limit  theorem  namely  that  if  Yn,Y  are  row  vector 

valued  and  Z  matrix-valued  random  variables  such  that  when  n  -»  °o, 
n 
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^  Y,  p  lim  Z^  =  C,  then  Y^Z^  ^  YC  (^  denoting  convergence  in 
law).,  whenever  the  products  are  defined.  (Compare,  for  example,  Rao, 
p.102  (xb)). 

(4.27.)  is  the  rigorous  statement  of  the  result  (1.5)  for  q=l 
and  g(u),  0)  =1  as  a  limit  theorem.  The  explicit  formula  for  the  inverse 
of  ¥  is 


(4.29) 


An  equivalent  way  of  stating  the  result  of  Theorem  3  is  therefore  that 

AAA 

the.  distribution  of  (A  ,B  ,wn)  is  asymptotically  normal  with  mean 
(0,0,0)  and  covariance  matrix 


2v 


2  2 

VBo 


-5“'Vo 

-6n“2B„ 


-3“'Vo 


6n~2A 


0 


-6n"2B 
6n“2A, 


12  n 


(4.30) 


The  most  notable  feature  of  (4.30)  is  the  very  rapid  decrease  of 

z'*  "5  2  2 

the  asymptotic  variance  of  <*)n,  2kv/{rP  (A^+Bq)  ),  when  n  increases. 
In  fact  in  maximum  likelihood  estimation  it  is  very  rare  to  obtain  an 

_3 

asymptotic  variance  of  order  n  ,  although  asymptotic  variances  of 
-2 

n  are  not  unusual.  This  phenomenon  is  due  to  the  sharpness  of  the 
largest  peak  of  the  periodogram  intensity  function  I  .  We  might  con- 

/S  /\ 

sequently  expect  the  asymptotic  variances  of  A  and  B  to  be  the 

n  n 

same  as  if  the  frequency  were  known,  but  we  see  from  (4.30)  that 
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this  is  not  so.  For  example ,  the  asymptotic  variance  of  A^  is 
2  2  2  2 

2v(Aq+  AB^/C^Aq+Bq) },  -.which  is  greater  than  2v/n,  the  asymptotic 
2  n 

variance  of  —  S ,  .  X,  cos  w  t .  unless  Brt  =  0. 

n  t=l  tO  0 
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5.  The  case  of  several  harmonic  components, 
Suppose  now  that 


q. 


m,  =  E(X  )  =  Y  (A  cos  a)  t+B  sin  to  t)  ,  (5.1) 

t.  v  t'  r.,0  r,0  r,0  r,0  3  ' 


q,  q  and  <o^  ^  denoting  the  true  values  of  the  parameters.  The 
function  corresponding  to  (2.2)  whose  minimisation  yields  estimators 

/N  /S  /% 

A  ,  B  ,  0)  ■  (l  <  r  <  q)  then  becomes 

r,n  r,n'  r,n  —  —  ^ 


C0  -  2n 


-1 


<1  n 

r=l  t=l 


X,(A  cos  to  t+B  sin  to  t)  +  — 

■r  ®  *£>  no  -v*  -v»  '  '  J 


r  r 


^  (A^+B^)  ...  (5-2) 


Thus 


r  ,n 


2  £ 

—  >  X,  cos  io  t  , 

n  X--  t  r.n 

t=l  3 


/%  a  A 

B  =  -  V  X.  sin  to  t  ,  (5-3) 

r.n  n  /->  t  r.n  K 

3  t=l  3 


and  if  we  write 


•  ^to)^  ^0  —  ( C0_  )  (xt  )  .  •  .  j  to  )  j 

7  q  -n  v  l,n'  2,n  '  q,n' 


)  (o>)  =  7  I  (co  ) 

n^—  Lj .  n  r 

r=l 


(5-4) 


is  a  maximum  when  00  =  00 


■n 


Here,  however,  since  terms  of  the  form  A  A  ,  cos  w  t  cos  io  t 

r  s  t=l  r  s 

n 

and  B  B  E,,  sin  (0  t  sin  00  t  have  been  dropped  in  obtaining  (5-2) 

±  S  Ti— J-  I*  S 

from  the  residual  sum  of  squares 


n 


X  (Xt  -  X,  (Ar  008  “rt  +  Br  sin  “rt)!  ' 

t=l  r=l 


the  maximisation  of  (5*4)  cannot  be  unrestricted;  some  condition  must 
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be  imposed  to  keep  the  to  from  being  too  close  together  and  so  prevent 
two  angular  frequency  estimators  from  converging  in  probability  to  the 
same  value.  In  fact  unrestricted  maximisation  obviously  makes  the 
Ur  n  a-^  elua^  the  an6ular  frequency  for  which  1^  attains  its 
absolute  maximum,  and  this  will  converge  in  probability  to  the  to 

r,  0 

2  2  l/2 

for  which  the  corresponding  amplitude  (A^  ^  +B^  '  is  largest. 

The  required  condition  is 


lim  min  n|to  -to  |  =  00  (5-5) 

n  00  1  <r^s  <q 


We  might  therefore,  for  example,  minimise  (5*4)  subject  to 


min '  | 
r  f  s  • 


(5.6) 


When  (5*5)  holds,  then  in  the  relevant  domain,  say,  in  to 


space  of  the  function  cpn,  only  q  of  the  q 

-1 


differences  to  -  to  . 

s  r,0 


can  be  0(n  ) .  If  we  label  the  components  of  the  argument  of  cp  so 


that  these  differences  are  to  -  to  we  see  that  the  behaviour  of 

r  r  ,Cr 


incPn(H)  -  V  if  tDs,0Mn(“r+“s,0)  +  Ds,0Mn(“r-“B,0)+I1  V 
r=l  s=l  7  ’  ’  ’  t=l 


iio  t  „ 
r  1 2 


where 


D 


1 


s,0 


(A 


s,0 


i  B  ) 
s,Cr 


is  controlled  by  the  sum  of  terms 
(l  <  r  <  q)  are  small.  In  fact, 
we  take  a  sequence  of  sets  {Sr } 


%o  =  f  (VotlBS;o) 


when  to  -to  ^ 
r  r,0 


|d*  „M  (to  -to  J  |2 
r,0  n  r  r,0 

we  can  show,  just  as  in  §2,  that  if 
for  which  (5*5)  holds,  then 
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max 
co  e  S 


n 


2,  (Ar,0+Br,0>  lMn<V“r,0> 

r=l  ’ 


=  0  (n'^) 

P 


It  -will  follow  that  if  R  „  =  {co;  |co  -co  |  <n  1  <  r  <  q}  is  cori¬ 

tained  in  ■  S  ,  which  will  be  true  for  sufficiently  large  n,  and 
(c) 

R  4  =  S  -  R  _  denotes  its  complement  with  respect  to  S  ,  then  for 
n,o  n  n,S  n’ 

sufficiently  small  8, 


p  '  lim  {n 
n  ->  oo 


-1 


max  cp  (co) } 

>(c)  n~ 


co  e  R 


8  &  (Ar,0+Br,0>  l-T 


1  A  2 
sin  —  8V 


n,B 


(5*7) 


From  (5.7)  it  is  easily  deduced  that 


p  lim-  n(co  -  co  ) 
r,n  r,0 
n  -*  00  '  ’ 


=  1 


(5.8) 


.  Since  cp^  is  symmetrical  in  its  q  arguments,  a  means  of  deter- 

/s 

mining  which  component  of_  co  is  associated  with  a  particular  frequency 
has  to  be  found.  We  can,  however,  obtain  this  from  the  fact  that 


“1  ^ 

p  lim  n  I  (co  ) 
n  r,n 

n  -»  00 


I  (4,o o'*  > 


(5-9) 


which  is  fairly  readily  demonstrated  by  using  Taylor's  Theorem  and  (5*8) 

If  therefore  the  co  are  labelled  so  that 

r,0 

A(r  +B(r  >  +B^  >  ...  >  A^  +B ^  _,  then  with  probability  tending 

1,0  1,0  —  2,0  2,0  —  —  q,0  q,0'  0 

to  unity  as  n  -»  00, 


I  (co  )  >  I  (co  )>...>  I  (co  ) 
n  l,n'  —  n  2,n'  —  —  nv  q,n' 


Thus  if  we  determine  the  <0  as  the  q  largest  local  maxima  of  the 


r.n 


periodogram  intensity  subject  to  a  separation  condition  satisfying  (5«5) 
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these  will,  for  sufficiently  large  n,  almost  certainly  estimate  the 
frequencies  of  the  harmonic  components  arranged  in  descending  order  of 
magnitude . 

We  then  deduce  that 


P 


lim 

n  ->  oo 


A 


r  ,n 


p  lim 
n  oo 


B 

r,  n 


by  an  argument  of  the  type  used  in  §3 . 

Finally,  if  we  denote  (5*2)  multiplied  by  n  by  Un(A,B,to), 
where  A  =  (A^A^,  . .  „  ,A^) ,  B  =  (B-^Bg,  . . .  ,B^) ,  (compare  equation 
(4.1)) ,  we  see  that  ^  (A,B,<o)  is  of  the  form 


n  _  n  / 

5'  X2  +'  V  f  (A  ,B  ,o)  ,X^n') 
t  ^  n,r'  r’  t’  r  —  ' 

t=l  ’ 


r=l 


} 


so  that  applying  the  mean  value  theorem  as  in  §4  gives  us  q  sets  of 
equations  each  of  the  form  (4.2)  -  (4.4),  namely 


(n1/2(A  -A  ), 
r,n  r,0 


1/2 /£ 

n  (B 


J/2 


(to  -to  )  } 
'  r,n  r,0' 


.(n-l/2(Un)A  ,  n_l/2(Un)B  >  n"^2(Uj,  )  (W£  J_±  ,  (5-10) 

r,0  r,0 


n  to  ’  '  r,n# 
r,0 


in  an  obvious  notation.  From  results  of  the  type  (4.6)  -  (4.8),  for 
example , 


<Va 


r  ,0 


cos  to  t  +  0(l) 
r,  0 


and  an  application  of  the  central  limit  theorem  as  in  §4,  it  follows 
that  the  row  vectors  on  the  right-hand  sides  of  (5*10)  are  asymptotically 
distributed  independently  as  N(0,  2vW  ),  where  the  matrix  W  is 


-29- 


obtained  by  replacing  A^,Bq  in  "the  expression  (4.14)  for  W  respec¬ 
tively  by  A_^  0,Br  q.  Again  as  in  §4,  we  can  show  that 


p  lim  W* 

r,n 

n  -»  oo  } 


=  W 


and  so  we  reach  the  conclusion  that  the  row  vectors  on  the  left-hand 

sides  of  (5*10)  are  asymptotically  distributed  independently  as 
r-U 


N(0,  2vW  ) .  This  is  the  required  generalisation  of  Theorem  5' 


In  practice  the  determination  of  the  estimators  to 


r  ,n 


could  be 


very  troublesome  because  of  the  difficulty  of  maximising  cp^w)  sub¬ 
ject  to  a  restriction  such  as  (5*6)  and  the  awkward  problem  of  the' 
appropriate  choice  of  an  appropriate  minimum  separation  to  be  used  for 
a  particular  set  of  data  wpuld  also  arise.  We  shall  not  consider  such 
questions  here  as  our  purpose  is  restricted  to  the  rigorous  derivation 

of  Asymptotic  properties.  We  note,  however,  that  an  asymptotically 

>\ 

equivalent  procedure  would  be  to  determine  ^  by  maximising  I^co) 

unconditionally,  then  u)  by  maximising  unconditionally 

2 ,  n 

(X,  -  A  ■  cos  cm  t-B  sin  w  t)  eiU^|^,  where  A,  ,Bn 
'  n=l  vt  l,n  l,n  l,n  l,n  '  1  ’  ,  l,n'  l,n 

A 

are  obtained  from  (5-5),  and  so  on,  co  being  finally  determined  by 

q,n 

maximising 


q-1 


I  «t  -  E  <*r, 

n=l  r=l  ’ 


n 


cos  co  t  +  B  sin  co  t)}  e 

V»  vo  v»  to  -r*  y\  *  * 


icot  1 2 


r,n 


r,n 


r  ,n 
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